REPORT  DOCUMENTATION  PAGE 


Form  Approved  OMB  NO.  0704-0188 


The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions, 
searching  existing  data  sources,  gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments 
regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggesstions  for  reducing  this  burden,  to  Washington 
Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington  VA,  22202-4302. 
Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  oenalty  for  failing  to  comply  with  a  collection 
of  information  if  it  does  not  display  a  currently  valid  OMB  control  number. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


2.  REPORT  TYPE 
Final  Report 


1.  REPORT  DATE  (DD-MM-YYYY) 

21-08-2015 


4.  TITLE  AND  SUBTITLE 

ARO  Final  Report:  Coordination  and  Collective  Decision 
Making  W91 1NG-1 1-1-0385 


3.  DATES  COVERED  (From  -  To) 

17-Aug-201 1  -  15-May-2015 


5a.  CONTRACT  NUMBER 

W911NF-1 1-1-0385 


5b.  GRANT  NUMBER 


6.  AUTHORS 

Simon  A.  Levin,  Naomi  E.  Leonard,  Iain  D.  Couzin 


5c.  PROGRAM  ELEMENT  NUMBER 
611102 


5d.  PROJECT  NUMBER 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 


7.  PERFORMING  ORGANIZATION  NAMES  AND  ADDRESSES 

Princeton  University 
PO  Box  36 

87  Prospect  Avenue,  Second  Floor 

Princeton,  NJ  08544  -2020 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS 
(ES) 

U.S.  Army  Research  Office 
P.O.Box  12211 

Research  Triangle  Park,  NC  27709-2211 


8.  PERFORMING  ORGANIZATION  REPORT 
NUMBER 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 
ARO 


11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

60017-MA.66 


12.  DISTRIBUTION  AVAIL1B1LITY  STATEMENT 
Approved  for  Public  Release;  Distribution  Unlimited 


13.  SUPPLEMENTARY  NOTES 

The  views,  opinions  and/or  findings  contained  in  this  report  are  those  of  the  author(s)  and  should  not  contrued  as  an  official  Department 
of  the  Army  position,  policy  or  decision,  unless  so  designated  by  other  documentation. 


14.  ABSTRACT 

This  project  studied  individual  search  behavior  and  coordination  of  large  groups  of  individual  agents,  and  how 
consensus  is  achieved  in  decision-making.  Animal  groups  frequently  display  highly  coordinated  movements,  and 
provide  an  excellent  vehicle  by  which  to  understand  general  principles  that  underlie  collective  behavior.  We 
studied  collective  movement  in  animal  populations,  and  the  relationship  between  individual  decision  rules  and 
emergent  collective  patterns,  especially  when  different  individuals  have  conflicting  infonnation.  We  expanded 


15.  SUBJECT  TERMS 

collective  motion,  collective  decision  making,  information  transfer,  consensus,  search  algorithms 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 

15.  NUMBER 

a.  REPORT 

b.  ABSTRACT 

c.  THIS  PAGE 

ABSTRACT 

OF  PAGES 

UU 

UU 

UU 

UU 

Simon  Levin 


19b.  TELEPHONE  NUMBER 
609-258-6880 


Standard  Fonn  298  (Rev  8/98) 
Prescribed  by  ANSI  Std.  Z39.18 


Report  Title 

ARO  Final  Report:  Coordination  and  Collective  Decision  Making  W91 1NG-1 1-1-0385 

ABSTRACT 

This  project  studied  individual  search  behavior  and  coordination  of  large  groups  of  individual  agents,  and  how  consensus  is  achieved  in 
decision-making.  Animal  groups  frequently  display  highly  coordinated  movements,  and  provide  an  excellent  vehicle  by  which  to 
understand  general  principles  that  underlie  collective  behavior.  We  studied  collective  movement  in  animal  populations,  and  the  relationship 
between  individual  decision  rules  and  emergent  collective  patterns,  especially  when  different  individuals  have  conflicting  information.  We 
expanded  classical  search  models,  extending  multi-armed  bandit  and  other  Bayesian  approaches  to  information  gathering.  We  investigated 
the  role  of  communication  topology  in  generic  models  of  group  motion  and  decision-making,  with  particular  reference  to  optimizing 
performance  metrics  such  as  speed,  accuracy  and  robustness  of  decisions  to  intrinsic  or  extrinsic  sources  of  error.  Central  was  the  degree  to 
which  collective  optimization  can  be  approximated  in  situations  where  individuals  operate  for  individual  benefit  rather  than  group  success. 
We  employ  a  comprehensive  approach  using  mathematical  models  of  collective  foraging  (self-propelled  interacting  particles)  within  an 
evolutionary  framework,  and  tools  from  dynamical  systems  theory,  statistical  mechanics,  adaptive  dynamics  and  state-of-the-art 
computational  hardware  and  algorithms.  We  used  large-scale  individual-based  simulation  and  mathematical  analysis  under  biologically 
realistic  assumptions,  yet  the  results  can  help  us  elucidate  fundamental  biological  principles  that  can  be  relevant  to  a  wide  range  of  scales 
and  species  from  social  bacteria  to  large  mammals. 
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“Collective  Motion,  Collective  Decision-Making,  and  Collective  Action:  From  Microbes  to  Societies,”  MASpread/Rapid  Trade  Meeting, 
University  of  Arizona,  Tempe,  AZ  (March) 

“Channeling  Luca  Pacioli:  Multi-Disciplinarity  and  a  Sustainable  Future,”  Lecture  Given  on  Receiving  the  Luca  Pacioli  Prize,  Ca’Foscari 
University  of  Venice,  Italy  (March) 

“Dealing  with  Public  Goods  and  Common  Pool  Resources,”  The  IGB  Colloquium,  Leibniz-Institute  of  Freshwater  Ecology  and  Inland 
Fisheries,  Berlin,  Germany  (March) 

Keynote  Lecture.  “Collective  Motion,  Collective  Decision-Making  and  Collective  Action:  From  Microbes  to  Societies.”  Deutsche 
Physikalische  Gesellschaft  Spring  Meeting.  Physics  of  Socio-Economic  Systems  Division,  Berlin,  Germany  (March) 

“Evolutionary  Perspectives  on  Strategy,”  Business  Strategy  Interfaces  and  Frontiers,  PRISM  Foundation,  New  York,  NY  (May) 

“Cooperation  in  the  Global  Commons,”  NetSci  -  International  School  and  Conference  on  Complex  Networks,  La  Herradura,  Spain  (June) 

Number  of  Presentations:  32.00 

Non  Peer-Reviewed  Conference  Proceeding  publications  (other  than  abstracts): 


Received  Paper 


TOTAL: 


Number  of  Non  Peer-Reviewed  Conference  Proceeding  publications  (other  than  abstracts): 

Peer-Reviewed  Conference  Proceeding  publications  (other  than  abstracts): 


Received  Paper 


04/24/2014  53.00  Vaibhav  Srivastava,  Paul  Reverdy,  Naomi  E.  Leonard.  On  optimal  foraging  and  multi-armed  bandits, 
51st  Annual  Allerton  Conference  on  Communication,  Control  and  Computing.  02-OCT-13,  .  :  , 

04/24/2014  54.00  Vaibhav  Srivastava,  Naomi  E.  Leonard.  On  the  speed-accuracy  trade-off  in  collective  decision  making, 
Proceedings  of  the  52nd  IEEE  Conference  on  Decision  and  Control,  Florence  Italy.  10-DEC-13,  .  :  , 

04/24/2014  55.00  Naomi  Ehrich  Leonard,  Alex  Olshevsky.  Cooperative  learning  in  multi-agent  systems  from  intermittent 
measurements, 

52nd  IEEE  Conference  on  Decision  and  Control,  Florence,  Italy.  10-DEC-13,  .  :  , 

08/18/2015  82.00  Paul  Reverdy,  Naomi  E.  Leonard.  Satisficing  in  Gaussian  bandit  problems, 

53rd  IEEE  Conference  on  Decision  and  Control,  Los  Angeles,  CA.  ,  .  :  , 

08/18/2015  84.00  Naomi  Ehrich  Leonard.  Multi-Agent  System  Dynamics:  Bifurcation  and  Behavior  of  Animal  Groups, 
IFAC  Symposium  on  Nonlinear  Control  Systems  (NOLCOS),  Toulouse,  France.  ,  .  :  , 

08/18/201 5  83.00  William  Scott,  Naomi  Ehrich  Leonard.  Dynamics  of  Pursuit  and  Evasion  in  a  Heterogeneous  Herd, 

53rd  IEEE  Conference  on  Decision  and  Control,  Los  Angeles,  CA.  ,  .  :  , 

08/19/2015  86.00  Vaibhav  Srivastava,  Paul  Reverdy,  Naomi  E.  Leonard.  Surveillance  in  an  Abruptly  Changing  World  via 
Multiarmed  Bandits, 

53rd  IEEE  Conference  on  Decision  and  Control,  Los  Angeles,  CA.  ,  .  :  , 

08/26/2014  59.00  Paul  Reverdy,  Vaibhav  Srivastava,  Naomi  E.  Leonard.  Algorithmic  models  of  human  decision  making  in 
Gaussian  multi-armed  bandit  problems, 

2014  European  Control  Conference  (EEC).  24-JUN-14,  .  :  , 


TOTAL: 


8 


Number  of  Peer-Reviewed  Conference  Proceeding  publications  (other  than  abstracts): 


(d)  Manuscripts 


Received  Paper 


02/18/2013  24.00 


03/12/2013  25.00 


03/12/2013  27.00 


03/13/2013  28.00 


07/31/2012  6.00 


07/31/2012  5.00 


07/31/2012  3.00 


08/01/2012  10.00 


08/01/2012  11.00 


08/01/2013  30.00 


08/01/2013  35.00 


08/01/2013  34.00 


Darren  Pais,  Naomi  E.  Leonard.  Adaptive  Network  Dynamics  andEvolution  of  Leadership  in  Collective 
Migration, 

Physica  D:  Nonlinear  Phenomena  (1 1  2012) 

Ariana  Strandburg-Peshkin,  Colin  R.  Twomey,  Nikolai  W.  Bode,  Albert  B.  Kao,  Yael  Katz,  Christos  C. 
loannou,  Brin  Rosenthal,  Colin  J.  Torney,  Haishan  Wu,  Simon  A.  Levin,  lain  D.  Couzin.  Visual  sensory 
networks  and  effectiveinformation  transfer  in  animal  groups, 

PNAS  (01  2013) 

Albert  B.  Kao,  Noam  Miller,  Colin  Torney,  Andrew  T.  Hartnett,  lain  D.  Couzin.  Collective  learning  and 
optimal  consensus  decisions  in  social  animal  groups, 

PNAS  (02  2013) 

Allison  Kolpas,  Michael  Busch,  Hong  Li,  lain  D.  Couzin,  Linda  Petzold,  Jeff  Moehlis.  How  the  spatial 
position  of  individuals  affects  their  influence  on  swarms:  A  numerical  comparison  of  two  popular  swarm 
dynamics  models, 

PLoS  ONE  (11  2012) 

Darren  Pais,  Carlos  H.  Caidedo-Nunez,  Naomi  E.  Leonard.  Hopf  bifurcations  and  limit  cycles  in 
evolutionary  network  dynamics, 

SIAM  J  on  Applied  Dynamical  Systems  (05  2012) 

loannis  Poulakakis,  Luca  Scardovi,  Naomi  E.  Leonard.  Node  classification  in  collective  evidence 
accumulation  toward  a  decision, 

Proceedings  of  the  51st  IEEE  Conference  on  Decision  and  Control  (03  2012) 

George  F.  Young,  Luca  Scardovi,  Andrea  Cavagna  ,  Irene  Giardina,  Naomi  E.  Leonard.  Starling  flock 
networks  manage  uncertainty  in  consensus  at  low  cost, 

PLoS  Computational  Biology  (07  2012) 

Allison  K.  Shaw,  lain  D.  Couzin.  The  evolution  of  movement  behavior  and  information  usage  in  seasonal 
environments, 

The  American  Naturalist  (07  2012) 

Kolbjorn  Tunstrom,  Yael  Katz,  Christos  C.  loannou,  Cristian  Huepe,  Matthew  J.  Lutz,  lain  D.  Couzin. 
Collective  states,  multistability  and  transitional  behavior  in  animal  groups, 

PLoS  Computational  Biology  (06  2012) 

Katherine  Fitch,  Naomi  Ehrich  Leonard.  Information  Centrality  and  Optimal  Leader  Selection  in  Noisy 
Networks, 

Proceedings  of  the  IEEE  Conference  on  Decision  and  Control  (03  2013) 

Christos  loannou,  Manvir  Singh,  lain  Couzin.  Potential  leaders  trade-off  group  cohesion  for  speed  and 
accuracy  in  fish  schools  , 

Proceedings  of  the  Royal  Society  B:  Biological  Sciences  (06  2013) 

Ariana  Strandburg-Peshkin,  Colin  R.  Twomey,  Nikolai  W.  Bode,  Albert  B.  Kao,  Yael  Katz,  Christos  C. 
loannou,  Brin  Rosenthal,  Colin  J.  Torney,  Haishan  Wu,  Simon  A.  Levin,  lain  D.  Couzin  .  Visual  sensory 
networks  and  effective  information  transfer  in  animal  groups, 

Current  Biology  (05  2013) 


08/01/2013  33.00  Alfonso  Perez-Escudero,  Noam  Miller,  Andrew  T.  Hartnett,  Simon  Gamier,  lain  D.  Couzin,  Gonzalo  G.  De 
Polavieja.  Estimation  models  describe  well  collective  decisions  among  three  options, 

PNAS  (05  2013) 

08/01/2013  32.00  Vaibhav  Srivastava,  Naomi  E.  Leonard.  On  the  Speed-Accuracy  Trade-off  in  Collective  Decision  Making, 
Proceedings  of  the  IEEE  Conference  on  Decision  and  Control  (11  2013) 

08/01/2013  31.00  Vaibhav  Srivastava,  Paul  Reverdy,  Naomi  E.  Leonard.  On  Optimal  Foraging  and  Multi-armed  Bandits, 
Proceedings  of  the  51st  Allerton  Conference  on  Communication,  Control,  and  Computing  (07  2013) 

08/02/2012  12.00  Vishwesha  Guttal,  Pawel  Romanczuk,  Stephen  J.  Simpson,  Gregory  A.  Sword,  lain  D.  Couzin. 
Cannibalism  can  drive  the  evolution  of  behavioral-phase  polyphenism  in  locusts, 

Ecology  Letters  (04  2012) 

08/02/2012  15.00  Simon  Gamier,  Tucker  Murphy,  Matthew  Lutz,  Edward  Hurme,  Simon  Leblanc,  lain  D.  Couzin.  Stability 
and  responsiveness  in  a  self-organizing  living  architecture, 

PLoS  Computational  Biology  (07  2012) 

08/02/2012  14.00  C.C.  loannou,  V.  Guttal,  I.D.  Couzin.  Predatory  fish  select  for  coordinated  collective  motion  in  virtual  prey, 
Science  (01  2012) 

08/02/2012  13.00  Andrew  Berdahl,  Colin  J.  Torney,  Christos  C.  loannou,  Jolyon  J.  Faria,  lain  D.  Couzin.  Emergent  sensing 
of  complex  environments  by  social  animal  groups, 

Science  (06  2012) 

08/12/2015  71.00  Vaibhav  Srivastava,  Paul  Reverdy,  Naomi  Ehrich  Leonard.  Correlated  Multiarmed  Bandit  Problem: 
Bayesian  Algorithms  and  Regret  Analysis, 

Automatica  (07  2015) 

08/18/2015  78.00  Andrew  T.  Hartnett,  Emmanuel  Schertzer,  Simon  A.  Levin,  lain  D.  Couzin.  Heterogeneous  Preference  and 
Local  Nonlinearity  in  Consensus  Decision-Making, 

Physical  Review  Letters  (06  2015) 

08/18/2015  80.00  loannis  Poulakakis  ,  Luca  Scardovi,  Naomi  Ehrich  Leonard  Fellow,  George  F.  Young.  Information 
Centrality  and  Ordering  of  Nodes  forAccuracy  in  Noisy  Decision-Making  Networks, 

IEEE  TRANSACTIONS  ON  Automatic  Control  (to  appear  March  2016)  (10  2012) 

08/18/2015  77.00  Andrew  M.  Hein,  Sara  Brin  Rosenthal,  George  I.  Hagstrom,  Andrew  Berdahl,  Colin  J.  Torney,  lain  D. 

Couzin.  The  evolution  of  distributed  sensing  and  collective  computation  in  animal  populations, 

PNAS  (07  2015) 

08/18/2015  79.00  Andrew  M.  Hein L,  Simon  A.  Levin.  Physical  limits  on  bacterial  navigation  in  dynamic  environments, 
Physical  Review  Letters  (07  2015) 

08/21/2015  87.00  Katherine  Fitch,  Naomi  Ehrich  Leonard.  Joint  Centrality  Distinguishes  Optimal  Leaders  in  Noisy  Networks, 
Proceedings  of  the  52nd  IEEE  Conference  on  Decision  and  Control,  Florence,  Italy  (07  2014) 

08/26/2014  60.00  Colin  J.  Torney,  Tommaso  Lorenzi,  lain  D.  Couzin,  Simon  A.  Levin.  Information  processing  and  the 
evolution  of  unresponsiveness  in  collective  systems, 

Journal  of  the  Royal  Society  Interface  (07  2014) 

08/26/2014  63.00  Katherine  Fitch,  Naomi  E.  Leonard.  Joint  centrality  distinguishes  optimal  leaders  in  noisy  networks, 

IEEE  Transactions  on  Control  of  Network  Systems  (07  2014) 

08/26/2014  64.00  George  F.  Young,  Luca  Scardovi,  Naomi  E.  Leonard.  A  New  Notion  of  Effective  Resistance  for 
DirectedGraphs — Part  I:  Definition  and  Properties, 

IEEE  TRANSACTIONS  ON  Automatic  Control  (10  2013) 

08/26/2014  65.00  George  F.  Young,  Luca  Scardovi,  Naomi  E.  Leonard.  A  New  Notion  of  Effective  Resistance  for 
DirectedGraphs — Part  II:  Computing  Resistances, 

IEEE  TRANSACTIONS  ON  Automatic  Control  (10  2013) 


09/21/2012  16.00  Ugo  Lopez,  Jacques  Gautrais,  lain  D.  Couzin,  Guy  Theraulaz.  From  behavioral  analyses  to  models  of 
collective  motion  in  fish  schools, 

Interface  Focus  (06  2012) 

09/21/2012  17.00  Naomi  Ehrich  Leonard,  Alex  Olshevsky.  Cooperative  learning  in  multi-agent  systems  from  intermittent 
measurements, 

SIAM  J  of  Control  and  Optimization  (09  2012) 
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Naomi  Leonard 


Awards 


2014 

Inaugural  Glenn  L.  Martin  Medal,  A.  James  Clark  School  of  Engineering,  University  of  Maryland 
Nyquist  Lecture  Award,  Dynamic  Systems  and  Control  Division,  ASME 

Keynote  Lecture,  International  Federation  of  Automatica  Control  (IF AC)  World  Congress,  Cape  Town,  South  Africa 

Simon  A.  Levin 

2014 

Tyler  Prize  for  Environmental  Achievement 
Luca  Pacioli  Prize,  Ca’Foscari  University  of  Venice,  Italy 
Foreign  Member,  Istituto  Lombardo,  Milan 
1IASA  Distinguished  Visiting  Fellow 

Keynote  Lecture,  German  Physical  Society,  Physics  of  Socio-Economic  Systems  Division 
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National  Academy  Member 
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Number  of  graduating  undergraduates  funded  by  a  DoD  funded  Center  of  Excellence  grant  for 
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scholarships  or  fellowships  for  further  studies  in  science,  mathematics,  engineering  or  technology  fields: . 0.00 
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FTE  Equivalent: 

Total  Number: 


Sub  Contractors  (DD882) 


Inventions  (DD882) 


Scientific  Progress 


Research  Accomplishments:  Overview 


This  is  the  final  report  for  this  project,  which  has  produced  more  than  50  publications  on  how  consensus  is  achieved  in  large 
groups  of  independent  agents,  across  taxonomic  groups,  from  bacteria  to  humans.  Individual  reports  for  the  various  years  are 
attached  as  appendices.  We  have  studied,  theoretically  and  empirically,  the  high  degree  of  coordinated  movements  in  many 
animal  populations,  and  derived  general  principles  underlying  collective  behavior.  We  have  focused  on  scaling  from  individual 
interactions  to  collective  behavior,  and  back,  studying  the  role  of  communication  topologies,  and  conflicts  between  individual 
actions  and  collective  optimization.  We  feel  that  there  is  much  to  learn  from  animal  groups  more  generally  in  how  groups  can 
coordinate  decision-making. 

In  this  project,  we  studied  group  decision-making  in  the  face  of  conflicting  information,  and  in  particular  the  mechanisms  by 
which  collective  behaviors  emerge  in  nature.  Of  central  interest  is  the  degree  to  which  collective  optimization  can  be 
approximated  in  situations  where  individuals  operate  for  individual  benefit  rather  than  group  success.  Mathematical  and 
computational  models  are  central  to  our  investigations.  In  the  early  phases  of  this  work,  we  studied  how  a  group  may  be 
susceptible  to  manipulation  by  a  strongly  opinionated,  or  extremist,  minority.  Later  work  has  explored  this  in  greater  detail,  and 
the  latest  report  below  explores  this  in  depth. 

We  also  examined  the  dynamics  of  decision-making  among  finite  alternatives  in  large  networked  groups  using  the  replicator- 
mutator  equations  from  evolutionary  dynamics.  Details  may  be  found  in  the  individual  annual  reports. 

In  the  next  phase,  we  studied  information  transfer  and  processing  within  groups,  and  the  size  and  nature  of  the  network  from 
which  individuals  derive  information.  Again,  this  involved  integration  of  empirical  and  theoretical  studies  on  a  number  of  groups. 
We  also  focused  on  the  speed-accuracy  tradeoff  in  gathering  information,  and  on  how  individuals  effectively  mine  environments 
for  information.  Thus  we  explored  the  integration  of  individual  and  social  information,  how  this  emerges  in  animal  groups,  and 
how  it  can  inform  optimal  strategies  of  individual  and  collective  search. 

In  the  final  phase,  reported  in  the  pages  that  follow,  these  various  perspectives  are  integrated  into  coherent  overall  descriptions 
and  theories. 

Research  Accomplishments:  Final  Period 

1.  Collective  Foraging  in  Deep-Diving  Whales  (Matthieu  Barbier) 

Barbier  developed  a  model  of  whale  collective  foraging:  Agent-Based  simulations  and  analytical  tools  have  been  used  to  study 
the  importance  of  sonar  directionality  and  organized  motion  in  maximizing  the  detection  and  harvesting  of  prey  in  a  fast¬ 
changing  environment.  An  article  in  collaboration  with  postdoctoral  researcher  Frants  Jensen,  connecting  this  work  to  his 
empirical  work  on  pilot  whales  and  to  the  problem  of  the  evolution  of  sonar  directionality  in  various  species  is  in  progress. 

2.  The  Evolution  of  Distributed  Sensing  and  Collective  Computation  in  Animal  Populations  (George  Hagstrom,  Andrew  Hein, 
Sarah  Rosenthal,  Andrew  Berdahl,  Colin  Torney,  and  lain  Couzin) 

The  mechanism  by  which  social  interactions  among  organisms  result  in  coordinated  and  dynamic  collective  motion  has  been 
subject  to  extensive  exploration  in  recent  years.  While  collective  motion  occurs  at  a  macroscopic  level,  it  is  driven  by  real-time 
microscopic  level  movement  decisions  made  by  individuals.  These  individual  movement  rules  can  lead  to  rapid  transitions  in 
group  properties,  reminiscent  of  phase  transitions  in  physical  systems.  Importantly,  the  relationship  between  such  pattern 
formation  and  the  evolutionary  drivers  of  collective  behavior  are  poorly  understood.  It  is  unclear  what  decision  rules  lead  to 
efficient  collective  behavior,  and  precisely  how  such  rules  could  evolve  through  natural  selection  on  individual  behavior.  We 
have  investigated  these  phenomena  with  a  simple  model  of  social  decision-making,  based  on  experimental  studies  of  schooling 
fish,  in  which  agents  search  their  environment  for  a  resource  using  a  combination  of  social  and  environmental  information.  We 
developed  high  performance  numerical  simulations  and  theoretical  calculations  to  facilitate  both  the  discovery  of  the  parameters 
of  the  model  that  yield  optimal  group  performance,  as  well  as  evolutionary  simulations  to  study  the  rules  that  emerge  when  each 
agent  is  rewarded  based  on  its  individual  performance. 

The  numerical  study  revealed  that  evolved  populations  have  the  emergent  ability  to  sense  and  climb  dynamic  resource 
gradients.  This  occurs  even  though  individuals  have  no  direct  knowledge  of  the  number,  location,  or  size  of  the  resource 
patches.  The  agents  are  able  to  achieve  this  when  the  parameters  of  the  model  permit  the  existence  of  two  distinct  stable 
states,  a  dense  ‘liquid’  state  and  a  dilute  ‘gas’  state,  and  when  the  population  can  undergo  a  transition  between  states  near  the 
boundary  of  a  resource  path.  Density  differences  between  the  two  states  allow  agents  to  determine  the  location  of  resource 
patches  from  afar,  making  no  use  of  gradient  information.  This  study  is  the  first  we  are  aware  of  that  shows  that  natural 
selection  acting  on  selfish  individuals  can  lead  to  groups  with  effective  collective  computation  capabilities. 

To  support  our  numerical  simulations,  we  developed  a  continuum  description  of  the  movements  of  the  agents.  We  adapted 


closure  techniques  used  in  fluid  dynamics  and  other  branches  of  physics  to  the  topological  interaction  laws  obeyed  by  agents  in 
our  simulation,  and  derived  an  advection-diffusion  equation  for  the  density  of  agents.  This  enabled  us  to  find  a  mathematical 
formula  for  the  onset  of  phase  transitions  between  the  dense  and  dilute  states,  which  helped  us  verify  the  observation  that 
evolved  populations  took  advantage  of  phase  transitions  to  locate  and  exploit  research  patches.  We  were  able  to  model  this 
exploitation  process  and  demonstrate  that  social  agents  are  able  to  colonize  patches  at  an  exponential  rate,  while  asocial 
agents  are  only  able  to  colonize  them  at  a  linear  rate.  The  principles  gleaned  from  this  work  will  improve  understanding  of 
optimization  of  behavior  in  multi-agent  systems  and  have  the  potential  to  inform  design  of  control  protocols  for  autonomous 
vehicles  and  optimization  routines  to  be  used  in  online  optimization  tasks.  A  manuscript  from  this  work  is  complete  and  we  plan 
to  submit  it  to  the  journal  eLIFE. 

3.  Eulerian  Descriptions  of  Topologically  Interacting  Agents  (George  Hagstrom,  Simon  A.  Levin,  and  Glenn  Flierl) 

One  of  the  fundamental  problems  in  ecological  systems  is  to  understand  how  interactions  on  small  scales  lead  to  macroscopic 
dynamics  on  the  large  scales.  Large  scale  patterns  that  result  from  the  collective  behavior  of  biological  organisms  support 
consumers  and  predators  in  both  terrestrial  and  marine  food  webs,  as  well  as  enabling  interspecies  interactions  such  as 
reproduction.  Eulerian  models  describe  aggregations  of  animals  in  terms  of  local  variables  such  as  population  densities  and 
velocities,  and  these  models  have  been  used  to  understand  the  results  of  collective  interactions  of  all  types  of  physical  objects, 
from  molecules  to  animals.  Early  efforts  to  model  collective  behavior  were  based  on  metric  (distance)  based  interaction  rules 
that  were  more  inspired  by  ease  of  use  than  biological  relevance.  Indeed,  in  the  past  few  years  scientists  have  realized  that  the 
actual  interaction  laws  between  agents  cannot  usually  be  described  using  metric  interactions,  and  it  has  been  hypothesized  that 
other  rules  are  more  important. 

We  have  been  working  to  understand  the  implications  of  a  topological  interaction  law  (one  in  which  a  biological  agents  interacts 
with  nearest  neighbors)  on  collective  behavior.  Topological  interactions  have  been  suggested  to  be  more  realistic  than  metric 
ones,  and  there  is  experimental  evidence  that  they  describe  starling  flocks  and  other  important  model  organisms.  It  has  also 
been  suggested  that  the  topological  interaction  law  is  important  for  the  robustness  of  collective  behavior  and  that  it  facilitates 
easy  collective  decision-making.  We  have  derived  Eulerian  models  that  arise  from  an  individual  based  model  with  a  topological 
interaction  law,  including  a  kinetic  description  and  hierarchy  of  fluid  equations.  We  have  focused  on  analyzing  advection- 
diffusion  equations  resulting  from  closures  of  hierarchies  of  equations.  These  efforts  have  shown  a  number  of  different  things: 
some  common  features  of  metric  models,  such  as  collapse  in  non  H-stable  regimes,  do  not  occur  in  topological  models,  and  the 
effective  interaction  radius  is  dependent  on  the  local  density.  We  have  studied  both  local  and  global  properties  of  equilibria  of 
these  advection-diffusion  equations,  deriving  criteria  for  both  linear  stability/instability  and  conditions  for  global  attractiveness  of 
the  homogeneous  equilibrium  state.  This  work  has  been  coupled  with  numerical  studies  of  the  corresponding  individual  based 
models,  both  to  verify  the  accuracy  of  the  closure  scheme  and  to  investigate  important  high-level  properties,  such  as  group  size 
distributions  and  group  transition  graphs. 

4.  Physical  Limits  on  Bacterial  Navigation  in  Dynamic  Environments  (Andrew  Hein,  Simon  Levin) 

Andrew  Hein  led  a  project  in  which  we  (Hein,  Douglas  R.  Brumley,  Francesco  Carrara,  Roman  Stocker,  and  Simon  A.  Levin) 
developed  mathematical  tools  to  model  bacterial  navigation  in  complex  environments.  Many  biological  and  artificial  sensory 
systems  have  limited  accuracy;  the  physics  of  environmental  cues  create  noise  that  result  in  a  lower  bound  on  the  minimum 
signal  a  sensory  system  can  resolve.  We  showed  that  there  is  a  lower  bound  on  the  chemical  signals  that  bacterial  cells  can 
measure,  and  that  this  lower  bound  can  be  used  to  make  predictions  about  how  bacteria  navigate  noisy,  dynamic  environments. 
The  mathematical  tools  we  have  developed  are  general  and  apply  to  both  biological  and  artificial  systems.  We  have  submitted 
a  manuscript  resulting  from  this  study  to  Physical  Review  Letters.  We  are  continuing  to  develop  this  theoretical  framework  and 
working  on  a  second  manuscript. 

5.  Decision-making  Linder  Uncertainty  (Naomi  Leonard) 

In  Srivastava,  Reverdy,  and  Leonard  (Proc.  Conf.  Decision  and  Control,  December  2014),  we  have  extended  our  work  on 
modeling  and  analysis  of  individual  decision-making  under  uncertainty  in  foraging  to  environments  that  are  abruptly  changing 
due  to  the  arrival  of  unknown  spatial  events.  In  our  earlier  work  (Reverdy  et  al.,  Proc.  IEEE,  2014),  we  presented  a  formal 
model  of  Bayesian  inference  and  decision-making  in  explore-exploit  tasks  using  the  context  of  multi-armed  bandit  (MAB) 
problems,  where  the  decision-maker  must  choose  among  multiple  options  with  uncertain  (Gaussian)  rewards  with  the  goal  of 
maximizing  accumulated  reward.  We  proved  that  our  upper  credible  limit  (UCL)  algorithm  achieves  logarithmic  cumulative 
expected  regret,  which  is  optimal  performance  for  uninformative  priors,  and  we  showed  how  good  priors  and  good  assumptions 
on  the  correlation  structure  among  arms  can  greatly  enhance  decision-making  performance,  even  over  short  time  horizons. 

In  the  new  work,  we  studied  the  exploration-exploitation  tradeoff  in  MAB  problems  with  change  points.  In  such  problems,  the 
mean  rewards  from  arms  are  not  stationary,  and  may  abruptly  change  to  an  unknown  value  at  some  unknown  time.  We 
assume  that  the  order  of  the  number  of  possible  changes  within  time  T  is  known.  The  algorithm  we  propose  adapts  the  UCL 
algorithm  to  include  a  sliding  window  where  only  observations  in  this  recent  time-window  are  used  to  estimate  mean  rewards 


and  select  arms.  The  width  of  the  time-window  can  be  chosen  to  achieve  provably  efficient  performance.  We  also  extend  the 
algorithm  to  restrict  the  number  of  transitions  among  arms  by  incorporating  a  block  allocation  strategy. 

In  Reverdy  and  Leonard  (Proc.  Conf.  Decision  and  Control,  December  2014),  we  have  proposed  a  satisficing  objective  for  the 
MAB  problem,  where  the  objective  is  to  achieve  performance  above  a  given  threshold.  This  is  a  relaxation  of  the  maximizing 
objective  and  allows  for  less  risky  decision-making  in  the  face  of  uncertainty.  This  is  evident  in  the  algorithm  we  derive.  We 
find  a  bound  performance  for  this  algorithm. 

6.  Coordination  and  Communication  (Couzin) 

Coordination  among  social  animals  requires  rapid  and  efficient  transfer  of  information  among  individuals,  which  may  depend 
crucially  on  the  underlying  structure  of  the  communication  network.  Establishing  the  decision-making  circuits  and  networks  that 
give  rise  to  individual  behavior  has  been  a  central  goal  of  neuroscience.  However,  the  analogous  problem  of  determining  the 
structure  of  the  communication  network  among  organisms  that  gives  rise  to  coordinated  collective  behavior,  such  as  is  exhibited 
by  schooling  fish  and  flocking  birds,  has  remained  almost  entirely  neglected.  We  studied  collective  evasion  maneuvers, 
manifested  through  rapid  waves,  or  cascades,  of  behavioral  change  (a  ubiquitous  behavior  among  taxa)  in  schooling  fish 
(Notemigonus  crysoleucas).  We  automatically  tracked  the  positions  and  body  postures,  calculated  visual  fields  of  all  individuals 
in  schools  of  ~1 50  fish,  and  employed  statistical  and  machine-learning  methodologies  to  establish  how  these  organisms  map 
complex,  high-dimensional,  sensory  input  to  a  relatively  low-dimensional  behavioral  output  (escape  response)  during  collective 
evasion.  We  found  that  individuals  use  simple,  robust  measures  to  assess  behavioral  changes  in  neighbors,  and  that  the 
resulting  networks  by  which  behavior  propagates  throughout  groups  are  complex,  being  weighted,  directed,  and 
heterogeneous.  By  studying  these  interaction  networks,  we  revealed  the  (complex,  fractional)  nature  of  social  contagion  and 
established  that,  in  contrast  to  the  conventional  view  of  spreading  processes  that  individuals  with  relatively  few,  but  strongly 
connected,  neighbors  are  both  most  socially  influential  and  most  susceptible  to  social  influence.  Furthermore,  we  demonstrated 
that  we  can  predict  complex  cascades  of  behavioral  change  at  their  moment  of  initiation,  before  they  actually  occur. 
Consequently,  despite  the  intrinsic  stochasticity  of  individual  behavior,  establishing  the  hidden  communication  networks  in  large 
self-organized  groups  facilitates  a  quantitative  understanding  of  behavioral  contagion  (Rosenthal  et  al. ,  2015). 

Again  considering  the  network  of  interactions  among  organisms  we  analyzed  high  temporal  and  spatial  resolution  global 
positioning  system  (GPS)  data  of  a  study  of  collective  motion  in  baboons  (Strandburg-Peshkin  et  al.,  2015).  In  this  study, 
conducted  under  natural  field  conditions  in  Kenya,  we  obtained  the  positions  of  almost  all  adults  in  a  wild  troop,  every  second, 
for  approximately  three  weeks.  Baboons  are  highly  social  organisms  and  maintained  a  high  degree  of  cohesion,  thus  making 
movement  decisions  collectively  throughout  this  period  (such  as  when,  and  where  to  move).  In  contrast  to  the  fish  schools 
studied  in  Rosenthal  et  al.,  2015,  baboons  are  famous  for  living  in  complex,  socially-stratified  societies  with  a  prominent,  linear 
dominance  hierarchy.  Determining  how  collective  decision-making  operates  in  species  where  long-term  social  bonds  and 
dominance  hierarchies  introduce  asymmetries  in  individual  influence  remains  a  core  challenge  for  understanding  the  evolution 
of  social  complexity,  and  could  inform  engineered  systems.  Because  troops  are  heterogeneous  and  members  differ 
significantly  in  both  their  needs  and  capabilities,  conflicts  of  interest  over  movement  decisions  are  inevitable. 

We  developed  an  automated  procedure  for  extracting  movement  ‘initiation  attempts’  based  on  the  relative  movements  of  pairs 
of  individuals.  Using  this  procedure,  we  extracted  sequences  of  movements  in  which  one  individual  moved  away  from  another, 
and  was  either  followed  (defined  as  a  “pull”)  or  was  not  followed  and  subsequently  returned  (defined  as  an  “anchor”).  Multiple 
individuals  can  attempt  to  initiate  movement  at  the  same  time,  and  furthermore  they  can  do  so  in  different  directions.  Firstly  we 
found  that  neither  sex  nor  dominance  status  played  a  role  in  whether  individuals  were  followed,  and  thus  that  collective 
movement  decisions  in  baboons  result  from  a  shared,  democratic  process.  Secondly,  theoretical  studies  of  collective  decision¬ 
making  predict  that  when  two  subsets  of  individuals  have  conflicting  directional  preferences,  two  movement  regimes  will 
emerge:  If  the  difference  between  preferred  directions  (angle  of  disagreement)  is  relatively  small,  individuals  in  groups  are 
expected  to  “compromise”  by  moving  in  the  average  of  these  proposed  directions.  However,  above  a  critical  angle,  individuals 
“choose”  adopting  one  travel  direction  or  the  other  (Couzin  et  al.,  2005;  see  Figure  1,  A).  Consistent  with  this  theory,  baboon 
followers  clearly  exhibited  these  two  regimes.  In  the  simplest  case  of  two  concurrent  initiators,  the  fine-scale  movements  of 
baboons  demonstrated  that  followers  compromised  (exhibit  directional  averaging)  when  the  angle  of  disagreement  was  small, 
but  when  angles  exceeded  approximately  90  degrees,  followers  consistently  chose  one  direction  or  the  other  (Figure  1  B). 

When  initiators  propose  strongly  conflicting  directions,  how  do  followers  choose  which  direction  to  move  in?  As  predicted  by 
Couzin  et  al.,  2005,  we  found  that  when  choosing  between  competing  groups  of  initiators,  baboons  employ  a  simple  majority 
rule;  they  are  more  likely  to  follow  the  subgroup  containing  the  greatest  number  of  initiators.  As  the  asymmetry  in  initiator  group 
size  increases,  individuals  become  more  and  more  likely  to  follow  the  majority  (theoretical  predictions  shown  in  Figure  1C,  and 
experimental  data  from  wild  baboons  in  Figure  ID). 
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Figure  1 .  Comparison  of  theoretical  predictions  (A  and  C)  and  experimental  data  (B  and  D)  for  the  direction  taken  by  individuals 
as  a  function  of  the  angular  difference  between  potential  leaders  for  conditions  when  there  are  an  equal  number  in  each 
opposing  subset  (A  and  B)  and  when  the  lower  subset  (set  to  0  degrees)  is  in  the  majority  (C  and  D).  Modified  from  Couzin  et 
al.,  2005  (A  and  C)  and  Strandburg-Peshkin  et  al. ,  2015  (B  and  D). 
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Coordination  and  Collective  Decision  Making 
W91  INF-1 1-1  -0385 

Research  Accomplishments:  Overview 

This  is  the  final  report  for  this  project,  which  has  produced  more  than  50  publications  on  how  consensus 
is  achieved  in  large  groups  of  independent  agents,  across  taxonomic  groups,  from  bacteria  to  humans. 
Individual  reports  for  the  various  years  are  attached  as  appendices.  We  have  studied,  theoretically  and 
empirically,  the  high  degree  of  coordinated  movements  in  many  animal  populations,  and  derived  general 
principles  underlying  collective  behavior.  We  have  focused  on  scaling  from  individual  interactions  to 
collective  behavior,  and  back,  studying  the  role  of  communication  topologies,  and  conflicts  between 
individual  actions  and  collective  optimization.  We  feel  that  there  is  much  to  learn  from  animal  groups 
more  generally  in  how  groups  can  coordinate  decision-making. 

In  this  project,  we  studied  group  decision-making  in  the  face  of  conflicting  information,  and  in  particular 
the  mechanisms  by  which  collective  behaviors  emerge  in  nature.  Of  central  interest  is  the  degree  to 
which  collective  optimization  can  be  approximated  in  situations  where  individuals  operate  for  individual 
benefit  rather  than  group  success.  Mathematical  and  computational  models  are  central  to  our 
investigations.  In  the  early  phases  of  this  work,  we  studied  how  a  group  may  be  susceptible  to 
manipulation  by  a  strongly  opinionated,  or  extremist,  minority.  Later  work  has  explored  this  in  greater 
detail,  and  the  latest  report  below  explores  this  in  depth. 

We  also  examined  the  dynamics  of  decision-making  among  finite  alternatives  in  large  networked  groups 
using  the  replicator-mutator  equations  from  evolutionary  dynamics.  Details  may  be  found  in  the 
individual  annual  reports. 

In  the  next  phase,  we  studied  information  transfer  and  processing  within  groups,  and  the  size  and  nature 
of  the  network  from  which  individuals  derive  information.  Again,  this  involved  integration  of  empirical 
and  theoretical  studies  on  a  number  of  groups.  We  also  focused  on  the  speed-accuracy  tradeoff  in 
gathering  information,  and  on  how  individuals  effectively  mine  environments  for  information.  Thus  we 
explored  the  integration  of  individual  and  social  information,  how  this  emerges  in  animal  groups,  and 
how  it  can  inform  optimal  strategies  of  individual  and  collective  search. 

In  the  final  phase,  reported  in  the  pages  that  follow,  these  various  perspectives  are  integrated  into  coherent 
overall  descriptions  and  theories. 

Research  Accomplishments:  Final  Period 

1.  Collective  Foraging  in  Deep-Diving  Whales  (Matthieu  Barbier) 

Barbier  developed  a  model  of  whale  collective  foraging:  Agent-Based  simulations  and  analytical  tools 
have  been  used  to  study  the  importance  of  sonar  directionality  and  organized  motion  in  maximizing  the 
detection  and  harvesting  of  prey  in  a  fast-changing  environment.  An  article  in  collaboration  with 
postdoctoral  researcher  Frants  Jensen,  connecting  this  work  to  his  empirical  work  on  pilot  whales  and  to 
the  problem  of  the  evolution  of  sonar  directionality  in  various  species  is  in  progress. 

2.  The  Evolution  of  Distributed  Sensing  and  Collective  Computation  in  Animal  Populations  (George 
Hagstrom,  Andrew  Hein,  Sarah  Rosenthal,  Andrew  Berdahl,  Colin  Torney,  and  Iain  Couzin) 

The  mechanism  by  which  social  interactions  among  organisms  result  in  coordinated  and  dynamic 
collective  motion  has  been  subject  to  extensive  exploration  in  recent  years.  While  collective  motion 
occurs  at  a  macroscopic  level,  it  is  driven  by  real-time  microscopic  level  movement  decisions  made  by 
individuals.  These  individual  movement  rules  can  lead  to  rapid  transitions  in  group  properties, 
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reminiscent  of  phase  transitions  in  physical  systems.  Importantly,  the  relationship  between  such  pattern 
formation  and  the  evolutionary  drivers  of  collective  behavior  are  poorly  understood.  It  is  unclear  what 
decision  rules  lead  to  efficient  collective  behavior,  and  precisely  how  such  rules  could  evolve  through 
natural  selection  on  individual  behavior.  We  have  investigated  these  phenomena  with  a  simple  model  of 
social  decision-making,  based  on  experimental  studies  of  schooling  fish,  in  which  agents  search  their 
environment  for  a  resource  using  a  combination  of  social  and  environmental  information.  We  developed 
high  performance  numerical  simulations  and  theoretical  calculations  to  facilitate  both  the  discovery  of  the 
parameters  of  the  model  that  yield  optimal  group  performance,  as  well  as  evolutionary  simulations  to 
study  the  rules  that  emerge  when  each  agent  is  rewarded  based  on  its  individual  performance. 

The  numerical  study  revealed  that  evolved  populations  have  the  emergent  ability  to  sense  and  climb 
dynamic  resource  gradients.  This  occurs  even  though  individuals  have  no  direct  knowledge  of  the 
number,  location,  or  size  of  the  resource  patches.  The  agents  are  able  to  achieve  this  when  the  parameters 
of  the  model  permit  the  existence  of  two  distinct  stable  states,  a  dense  ‘liquid’  state  and  a  dilute  ‘gas’ 
state,  and  when  the  population  can  undergo  a  transition  between  states  near  the  boundary  of  a  resource 
path.  Density  differences  between  the  two  states  allow  agents  to  determine  the  location  of  resource 
patches  from  afar,  making  no  use  of  gradient  information.  This  study  is  the  first  we  are  aware  of  that 
shows  that  natural  selection  acting  on  selfish  individuals  can  lead  to  groups  with  effective  collective 
computation  capabilities. 

To  support  our  numerical  simulations,  we  developed  a  continuum  description  of  the  movements  of  the 
agents.  We  adapted  closure  techniques  used  in  fluid  dynamics  and  other  branches  of  physics  to  the 
topological  interaction  laws  obeyed  by  agents  in  our  simulation,  and  derived  an  advection-diffusion 
equation  for  the  density  of  agents.  This  enabled  us  to  find  a  mathematical  formula  for  the  onset  of  phase 
transitions  between  the  dense  and  dilute  states,  which  helped  us  verify  the  observation  that  evolved 
populations  took  advantage  of  phase  transitions  to  locate  and  exploit  research  patches.  We  were  able  to 
model  this  exploitation  process  and  demonstrate  that  social  agents  are  able  to  colonize  patches  at  an 
exponential  rate,  while  asocial  agents  are  only  able  to  colonize  them  at  a  linear  rate.  The  principles 
gleaned  from  this  work  will  improve  understanding  of  optimization  of  behavior  in  multi-agent  systems 
and  have  the  potential  to  inform  design  of  control  protocols  for  autonomous  vehicles  and  optimization 
routines  to  be  used  in  online  optimization  tasks.  A  manuscript  from  this  work  is  complete  and  we  plan  to 
submit  it  to  the  journal  eLIFE. 

3.  Eulerian  Descriptions  of  Topologically  Interacting  Agents  (George  Hagstrom,  Simon  A.  Levin,  and 
Glenn  Flier l) 

One  of  the  fundamental  problems  in  ecological  systems  is  to  understand  how  interactions  on  small  scales 
lead  to  macroscopic  dynamics  on  the  large  scales.  Large  scale  patterns  that  result  from  the  collective 
behavior  of  biological  organisms  support  consumers  and  predators  in  both  terrestrial  and  marine  food 
webs,  as  well  as  enabling  interspecies  interactions  such  as  reproduction.  Eulerian  models  describe 
aggregations  of  animals  in  terms  of  local  variables  such  as  population  densities  and  velocities,  and  these 
models  have  been  used  to  understand  the  results  of  collective  interactions  of  all  types  of  physical  objects, 
from  molecules  to  animals.  Early  efforts  to  model  collective  behavior  were  based  on  metric  (distance) 
based  interaction  rules  that  were  more  inspired  by  ease  of  use  than  biological  relevance.  Indeed,  in  the 
past  few  years  scientists  have  realized  that  the  actual  interaction  laws  between  agents  cannot  usually  be 
described  using  metric  interactions,  and  it  has  been  hypothesized  that  other  rules  are  more  important. 

We  have  been  working  to  understand  the  implications  of  a  topological  interaction  law  (one  in  which  a 
biological  agents  interacts  with  nearest  neighbors)  on  collective  behavior.  Topological  interactions  have 
been  suggested  to  be  more  realistic  than  metric  ones,  and  there  is  experimental  evidence  that  they 
describe  starling  flocks  and  other  important  model  organisms.  It  has  also  been  suggested  that  the 
topological  interaction  law  is  important  for  the  robustness  of  collective  behavior  and  that  it  facilitates  easy 
collective  decision-making.  We  have  derived  Eulerian  models  that  arise  from  an  individual  based  model 
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with  a  topological  interaction  law,  including  a  kinetic  description  and  hierarchy  of  fluid  equations.  We 
have  focused  on  analyzing  advection-diffusion  equations  resulting  from  closures  of  hierarchies  of 
equations.  These  efforts  have  shown  a  number  of  different  things:  some  common  features  of  metric 
models,  such  as  collapse  in  non  H-stable  regimes,  do  not  occur  in  topological  models,  and  the  effective 
interaction  radius  is  dependent  on  the  local  density.  We  have  studied  both  local  and  global  properties  of 
equilibria  of  these  advection-diffusion  equations,  deriving  criteria  for  both  linear  stability/instability  and 
conditions  for  global  attractiveness  of  the  homogeneous  equilibrium  state.  This  work  has  been  coupled 
with  numerical  studies  of  the  corresponding  individual  based  models,  both  to  verify  the  accuracy  of  the 
closure  scheme  and  to  investigate  important  high-level  properties,  such  as  group  size  distributions  and 
group  transition  graphs. 

4.  Physical  Limits  on  Bacterial  Navigation  in  Dynamic  Environments  (Andrew  Hein,  Simon  Levin) 

Andrew  Hein  led  a  project  in  which  we  (Hein,  Douglas  R.  Brumley,  Francesco  Carrara,  Roman  Stocker, 
and  Simon  A.  Levin)  developed  mathematical  tools  to  model  bacterial  navigation  in  complex 
environments.  Many  biological  and  artificial  sensory  systems  have  limited  accuracy;  the  physics  of 
environmental  cues  create  noise  that  result  in  a  lower  bound  on  the  minimum  signal  a  sensory  system  can 
resolve.  We  showed  that  there  is  a  lower  bound  on  the  chemical  signals  that  bacterial  cells  can  measure, 
and  that  this  lower  bound  can  be  used  to  make  predictions  about  how  bacteria  navigate  noisy,  dynamic 
environments.  The  mathematical  tools  we  have  developed  are  general  and  apply  to  both  biological  and 
artificial  systems.  We  have  submitted  a  manuscript  resulting  from  this  study  to  Physical  Review  Letters. 
We  are  continuing  to  develop  this  theoretical  framework  and  working  on  a  second  manuscript. 

5.  Decision-making  under  Uncertainty  (Naomi  Leonard) 

In  Srivastava,  Reverdy,  and  Leonard  (Proc.  Conf.  Decision  and  Control,  December  2014),  we  have 
extended  our  work  on  modeling  and  analysis  of  individual  decision-making  under  uncertainty  in  foraging 
to  environments  that  are  abruptly  changing  due  to  the  arrival  of  unknown  spatial  events.  In  our  earlier 
work  (Reverdy  et  al.,  Proc.  IEEE,  2014),  we  presented  a  formal  model  of  Bayesian  inference  and 
decision-making  in  explore-exploit  tasks  using  the  context  of  multi-armed  bandit  (MAB)  problems,  where 
the  decision-maker  must  choose  among  multiple  options  with  uncertain  (Gaussian)  rewards  with  the  goal 
of  maximizing  accumulated  reward.  We  proved  that  our  upper  credible  limit  (UCL)  algorithm  achieves 
logarithmic  cumulative  expected  regret,  which  is  optimal  performance  for  uninformative  priors,  and  we 
showed  how  good  priors  and  good  assumptions  on  the  correlation  structure  among  arms  can  greatly 
enhance  decision-making  performance,  even  over  short  time  horizons. 

In  the  new  work,  we  studied  the  exploration-exploitation  tradeoff  in  MAB  problems  with  change  points. 

In  such  problems,  the  mean  rewards  from  arms  are  not  stationary,  and  may  abruptly  change  to  an 
unknown  value  at  some  unknown  time.  We  assume  that  the  order  of  the  number  of  possible  changes 
within  time  T  is  known.  The  algorithm  we  propose  adapts  the  UCL  algorithm  to  include  a  sliding 
window  where  only  observations  in  this  recent  time-window  are  used  to  estimate  mean  rewards  and  select 
arms.  The  width  of  the  time-window  can  be  chosen  to  achieve  provably  efficient  performance.  We  also 
extend  the  algorithm  to  restrict  the  number  of  transitions  among  arms  by  incorporating  a  block  allocation 
strategy. 

In  Reverdy  and  Leonard  (Proc.  Conf.  Decision  and  Control,  December  2014),  we  have  proposed  a 
satisficing  objective  for  the  MAB  problem,  where  the  objective  is  to  achieve  performance  above  a  given 
threshold.  This  is  a  relaxation  of  the  maximizing  objective  and  allows  for  less  risky  decision-making  in 
the  face  of  uncertainty.  This  is  evident  in  the  algorithm  we  derive.  We  find  a  bound  performance  for  this 
algorithm. 

6.  Coordination  and  Communication  (Couzin) 

Coordination  among  social  animals  requires  rapid  and  efficient  transfer  of  information  among  individuals, 
which  may  depend  crucially  on  the  underlying  structure  of  the  communication  network.  Establishing  the 
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decision-making  circuits  and  networks  that  give  rise  to  individual  behavior  has  been  a  central  goal  of 
neuroscience.  However,  the  analogous  problem  of  determining  the  structure  of  the  communication 
network  among  organisms  that  gives  rise  to  coordinated  collective  behavior,  such  as  is  exhibited  by 
schooling  fish  and  flocking  birds,  has  remained  almost  entirely  neglected.  We  studied  collective  evasion 
maneuvers,  manifested  through  rapid  waves,  or  cascades,  of  behavioral  change  (a  ubiquitous  behavior 
among  taxa)  in  schooling  fish  ( Notemigonus  crysoleucas).  We  automatically  tracked  the  positions  and 
body  postures,  calculated  visual  fields  of  all  individuals  in  schools  of  ~150  fish,  and  employed  statistical 
and  machine-learning  methodologies  to  establish  how  these  organisms  map  complex,  high-dimensional, 
sensory  input  to  a  relatively  low-dimensional  behavioral  output  (escape  response)  during  collective 
evasion.  We  found  that  individuals  use  simple,  robust  measures  to  assess  behavioral  changes  in 
neighbors,  and  that  the  resulting  networks  by  which  behavior  propagates  throughout  groups  are  complex, 
being  weighted,  directed,  and  heterogeneous.  By  studying  these  interaction  networks,  we  revealed  the 
(complex,  fractional)  nature  of  social  contagion  and  established  that,  in  contrast  to  the  conventional  view 
of  spreading  processes  that  individuals  with  relatively  few,  but  strongly  connected,  neighbors  are  both 
most  socially  influential  and  most  susceptible  to  social  influence.  Furthermore,  we  demonstrated  that  we 
can  predict  complex  cascades  of  behavioral  change  at  their  moment  of  initiation,  before  they  actually 
occur.  Consequently,  despite  the  intrinsic  stochasticity  of  individual  behavior,  establishing  the  hidden 
communication  networks  in  large  self-organized  groups  facilitates  a  quantitative  understanding  of 
behavioral  contagion  (Rosenthal  et  al.,  2015). 

Again  considering  the  network  of  interactions  among  organisms  we  analyzed  high  temporal  and  spatial 
resolution  global  positioning  system  (GPS)  data  of  a  study  of  collective  motion  in  baboons  (Strandburg- 
Peshkin  et  al.,  2015).  In  this  study,  conducted  under  natural  field  conditions  in  Kenya,  we  obtained  the 
positions  of  almost  all  adults  in  a  wild  troop,  every  second,  for  approximately  three  weeks.  Baboons  are 
highly  social  organisms  and  maintained  a  high  degree  of  cohesion,  thus  making  movement  decisions 
collectively  throughout  this  period  (such  as  when,  and  where  to  move).  In  contrast  to  the  fish  schools 
studied  in  Rosenthal  et  al.,  2015,  baboons  are  famous  for  living  in  complex,  socially-stratified  societies 
with  a  prominent,  linear  dominance  hierarchy.  Determining  how  collective  decision-making  operates  in 
species  where  long-term  social  bonds  and  dominance  hierarchies  introduce  asymmetries  in  individual 
influence  remains  a  core  challenge  for  understanding  the  evolution  of  social  complexity,  and  could  inform 
engineered  systems.  Because  troops  are  heterogeneous  and  members  differ  significantly  in  both  their 
needs  and  capabilities,  conflicts  of  interest  over  movement  decisions  are  inevitable. 

We  developed  an  automated  procedure  for  extracting  movement  ‘initiation  attempts’  based  on  the  relative 
movements  of  pairs  of  individuals.  Using  this  procedure,  we  extracted  sequences  of  movements  in  which 
one  individual  moved  away  from  another,  and  was  either  followed  (defined  as  a  “pull”)  or  was  not 
followed  and  subsequently  returned  (defined  as  an  “anchor”).  Multiple  individuals  can  attempt  to  initiate 
movement  at  the  same  time,  and  furthermore  they  can  do  so  in  different  directions.  Firstly  we  found  that 
neither  sex  nor  dominance  status  played  a  role  in  whether  individuals  were  followed,  and  thus  that 
collective  movement  decisions  in  baboons  result  from  a  shared,  democratic  process.  Secondly,  theoretical 
studies  of  collective  decision-making  predict  that  when  two  subsets  of  individuals  have  conflicting 
directional  preferences,  two  movement  regimes  will  emerge:  If  the  difference  between  preferred 
directions  (angle  of  disagreement)  is  relatively  small,  individuals  in  groups  are  expected  to  “compromise” 
by  moving  in  the  average  of  these  proposed  directions.  However,  above  a  critical  angle,  individuals 
“choose”  adopting  one  travel  direction  or  the  other  (Couzin  et  al.,  2005;  see  Figure  1,  A).  Consistent  with 
this  theory,  baboon  followers  clearly  exhibited  these  two  regimes.  In  the  simplest  case  of  two  concurrent 
initiators,  the  fine-scale  movements  of  baboons  demonstrated  that  followers  compromised  (exhibit 
directional  averaging)  when  the  angle  of  disagreement  was  small,  but  when  angles  exceeded 
approximately  90  degrees,  followers  consistently  chose  one  direction  or  the  other  (Figure  1  B). 

When  initiators  propose  strongly  conflicting  directions,  how  do  followers  choose  which  direction  to  move 
in?  As  predicted  by  Couzin  et  al.,  2005,  we  found  that  when  choosing  between  competing  groups  of 
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initiators,  baboons  employ  a  simple  majority  rule;  they  are  more  likely  to  follow  the  subgroup  containing 
the  greatest  number  of  initiators.  As  the  asymmetry  in  initiator  group  size  increases,  individuals  become 
more  and  more  likely  to  follow  the  majority  (theoretical  predictions  shown  in  Figure  1C,  and 
experimental  data  from  wild  baboons  in  Figure  ID). 
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Figure  1.  Comparison  of  theoretical  predictions  (A  and  C)  and  experimental  data  (B  and  D)  for  the 
direction  taken  by  individuals  as  a  function  of  the  angular  difference  between  potential  leaders  for 
conditions  when  there  are  an  equal  number  in  each  opposing  subset  (A  and  B)  and  when  the  lower  subset 
(set  to  0  degrees)  is  in  the  majority  (C  and  D).  Modified  from  Couzin  et  ah,  2005  (A  and  C)  and 
Strandburg-Peshkin  et  ah,  2015  (B  and  D). 
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